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METHOD FOR PARTITIONING A PATTERN INTO OPTIMIZED SUB- 
PATTERNS 



FIELD OF THE INVENTION 

This invention relates to machine vision systems, and particularly to 
methods for partitioning a machine vision image. 

BACKGROUND OF THE INVENTION 

Searching for a particular pattern in an image is a well-known problem in 
the art of machine vision, with many known solutions, such as feature-based 
search methods. Typically, the pattern is assumed to have undergone one or 
more of a few basic known transformations, such as having been scaled larger or 
smaller than the original pattern, or having been rotated. However, these known 
solutions often fail if the pattern has been deformed by being warped, pulled, 
bent, wrinkled, damaged, or otherwise fundamentally changed from the original 
pristine shape, or known transformations thereof, that the search method is 
adapted to find. 
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However, if the deformed pattern is broken down into smaller sub- 
patterns, those individual sub-patterns are themselves fairly similar to the 
corresponding parts of the original pattern bearing only minor deformation. For 
example, if the pattern has been bent into a "V" or boomerang shape, then the 
two legs of the boomerang both have good, easily found pieces of the pattern. 
Therefore, searching for a deformed pattern in an image may be facilitated by 
dividing the pattern into smaller sub-patterns, because for many typical types of 
deformation encountered, most of those sub-patterns can probably be found by 
known feature-based search methods because they are not themselves 
substantially distorted or deformed. Then, a subsequent algorithm can combine 
these partial results into a full match of the pattern. 

The question is then how to divide the pattern into smaller sub-patterns. A 
human being can likely examine a large pattern, determine useful parts that will 
probably survive whatever deformation the image is expected to encounter, and 
divide it accordingly. However, automating the process is more difficult. 

The obvious and standard way to automatically divide a pattern image into 
sub-patterns is to use a rectilinear grid, such as a tic-tac-toe grid or a checker 
board grid. Super-imposing such a grid over the main pattern gives, for example, 
nine smaller sub-patterns. However, this method has significant drawbacks. 
Pattern matching is based on matching information in the source pattern with 
information in the target image. But useful information is not usually present 
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uniformly throughout a pattern. Some of the sub-patterns selected with a grid 
may be blank, and consequently have no useful information contained therein. 
Some grid-based sub-patterns may have a small amount of information in one 
corner of the grid square and no information in the rest of it. And some may have 
a lot of dense information that would be more usefully split into smaller pieces. 
Further, grids divide the information of a pattern arbitrarily and indiscriminately. 
If, for example, part of the pattern is a small star, the grid lines might break that 
star up into two or even four parts, where a human operator would likely choose 
to keep the star together as one single, dense, easily found sub-pattern. 

SUMMARY OF THE INVENTION 

In one general aspect, the invention provides a method for partitioning a 
pattern into optimized sub-patterns. The method includes providing a list of 
features of the pattern, then generating a set of candidate partitions using the list 
of features of the pattern. Each candidate partition of the set of candidate 
partitions is then scored, and a best-scoring partition among the set of candidate 
partitions is determined based on the scoring of each candidate partition. Then, 
the best scoring partition is applied to the list of features so as to provide a 
plurality of sub-lists of features respectively representing a plurality of optimized 
sub-patterns. 

In a preferred embodiment, providing a list of features includes using at 
least one sub-list from the plurality of sub-lists of features generated by an earlier 
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application of the method as the list of features of the pattern. In another 
preferred embodiment, providing a list of features of the pattern includes 
providing an image, and extracting a list of features from the image. In a further 
preferred embodiment, extracting a list of features from the images includes 
sampling the image so as to provide a regular array of pixels. In an alternate 
embodiment, extracting a list of features from the images includes using an edge 
extraction method to provide an edge image, and sampling the edge image to 
provide a plurality of edge feature points. In a further preferred embodiment, 
each edge feature point includes the angle of the edge at that edge feature point. 

In other preferred embodiments, features of the pattern are 2D image 
points. Alternatively, features of the pattern are points of any dimensionality. 

In yet other embodiments, providing a list of features includes providing an 
abstract pattern description, and extracting a list of features from the abstract 
pattern description. In alternate preferred embodiments, providing a list of 
features includes providing a pre-generated list of features. 

In another preferred embodiment, generating a set of candidate partitions 
using the list of features of the pattern includes using a clustering algorithm. In 
an alternate preferred embodiment, generating a set of candidate partitions using 
the list of features of the pattern includes using a spatial subdivision algorithm. In 
yet another embodiment, generating a set of candidate partitions using the list of 
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features of the pattern includes using a method that yields sub-lists that include 
pattern features that span an area of the pattern that is spatially small with 
respect to the area of the entire pattern. In still another preferred embodiment, 
generating a set of candidate partitions using the list of features of the pattern 
includes using a method that provides sub-lists having pattern features that are 
more near to each other than to pattern features in other sub-lists. In a further 
preferred embodiment, generating a set of candidate partitions using the list of 
features of the pattern includes building a weighted graph using the list of 
features of the pattern, and partitioning the weighted graph to generate candidate 
partitions. 

In a preferred embodiment, building a weighted graph using the list of 
features of the pattern includes fully connecting the feature points to make a 
graph, and assigning weights to each link. In an alternate preferred embodiment, 
building a weighted graph using the list of features of the pattern includes 
sparsely connecting the feature points to make a graph, and 
assigning weights to each link. In a further embodiment, the weight of each link 
is based on the distance between each pair of feature points. In other 
embodiments, weights decrease as the distance between feature points 
increases. In various other embodiments, the weights on each link are based on 
at least one of similarity of angle and similarity of magnitude. In yet other 
embodiments, the weights on each link are based on values associated with the 
feature points of the pattern. In still further embodiments, the weights on each 
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link are determined such that larger weights represent a pair of features that 
tend to be together in the same sub-lists of features, and smaller weights indicate 
a pair of features that can be included in different sub-lists of features. 

In some preferred embodiments, partitioning the weighted graph to 
generate candidate partitions includes dividing the weighted graph into two sub- 
graphs, one of which may be empty, and converting the two sub-graphs into two 
sub-lists of features. In other preferred embodiments, partitioning the weighted 
graph to generate candidate partitions includes partitioning the weighted graph 
using a "normalized cut" method to generate candidate partitions. 

In a preferred embodiment, when generating a set of candidate partitions 
using the list of features of the pattern, at least one candidate partition has only a 
single sub-list of features of the pattern. In an alternate embodiment, when 
generating a set of candidate partitions using the list of features of the pattern, 
each candidate partition has many sub-lists of features of the pattern. In yet 
another alternate embodiment, when generating a set of candidate partitions 
using the list of features of the pattern, some features included in the list of 
features of the pattern do not appear on any sub-list of features of the pattern. In 
still another embodiment, when generating a set of candidate partitions using the 
list of features of the pattern, at least one feature of the pattern appears on a 
plurality of sub-lists of features of the pattern. 
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In more preferred embodiments, scoring each partition of the set of 
candidate partitions includes building sub-patterns using the set of candidate 
partitions, and scoring each candidate partition using a scoring function based on 
characteristics of a sub-pattern derived therefrom. In a further embodiment, 
characteristics of the sub-pattern includes spatial coherence of the features 
corresponding to the sub-pattern. In an alternate further embodiment, 
characteristics of the sub-pattern includes overall spatial size of the area 
spanned by the feature points corresponding to the sub-pattern. In a preferred 
embodiment, the area spanned by the feature points is represented by the 
smallest bounding box that includes all the feature points. In another 
embodiment, characteristics of the sub-pattern includes the number of feature 
points in the sub-pattern. In yet another embodiment, characteristics of the sub- 
pattern includes the total amount of weight in links "cut" by the partition algorithm 
to create the sub-pattern. In still another embodiment, characteristics of the sub- 
pattern includes the overall "suitability" of the sub-pattern used as a search 
pattern applied to the original pattern. In still other embodiments, characteristics 
of the sub-pattern includes spatial coherence of the features corresponding to the 
sub-pattern, overall spatial size of the area spanned by the feature points 
corresponding to the sub-pattern, the number of feature points in the sub-pattern, 
the total amount of weight in links "cut" by the partition algorithm to create the 
sub-pattern, and the overall "suitability" of the sub-pattern used as a search 
pattern applied to the original pattern. 
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In some preferred embodiments, the overall "suitability" of the sub-pattern 
used as a search pattern applied to the original pattern depends on the search 
algorithm used. In other preferred embodiments, the overall "suitability" of the 
sub-pattern used as a search pattern applied to the original pattern depends on 
degeneracy of the features of a sub-pattern. In yet other preferred embodiments, 
the overall "suitability" of the sub-pattern used as a search pattern applied to the 
original pattern depends on redundancy of the sub-pattern within the original 
pattern. 

In a preferred embodiment, determining a best scoring partition based on 
the scoring includes using a partition score threshold. In some preferred 
embodiments, the partition score threshold is settable. In other preferred 
embodiments, the partition score threshold is predetermined. In yet other 
preferred embodiments, the partition score threshold includes a portion that is 
predetermined, and a portion that is settable. In a further embodiment, if no 
candidate partition has a score above the partition score threshold, then the list 
of features of the candidate partition is deemed to be one that cannot be usefully 
sub-divided. 

In another general aspect, the invention provides a method for 
automatically extracting a plurality of sub-patterns from a pattern in an image, the 
method including extracting a plurality of features, building a connected graph 
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using the plurality of features, and using the connected graph and a sub-division 
parameter to create a plurality of feature groups. 

In another general aspect, the invention provides a method for dividing a 
pattern into a plurality of sub-patterns, each sub-pattern being adapted for use 
with an image search method that can provide a plurality of sub-pattern search 
results. In this aspect, the method includes representing the pattern as a plurality 
of feature points, generating candidate partitions of the plurality of feature points, 
scoring the candidate partitions by examining characteristics of each potential 
sub-pattern of each candidate partition, selecting the highest-scoring partition, 
and then applying it to the plurlaity of feature points so as to create one or more 
sub-pluralities of feature points. In a preferred embodiment, the sub-pluralities of 
feature points are used as sub-patterns by an image search method that is 
adapted to use pluralities of feature points. In another preferred embodiment, the 
characteristics of each potential sub-pattern of each candidate partition include 
area, number of feature points, and suitability of the sub-pattern for use with a 
particular search method. 

The invention advantageously provides a plurality of sub-patterns where 
each sub-pattern contains a similar amount of information, where each sub- 
pattern has enough information to be located with a feature-based search 
method, and where that information has also been pre-evaluated as being useful 
and particularly adapted for running feature-based searches. 
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BRIEF DESCRIPTION OF THE DRAWING 

The invention will be more fully understood by reference to the detailed 
description, in conjunction with the following figures, wherein: 

Fig. 1a is a depiction of an exemplary sampling of a pattern image 
consisting of a set of 2D pixels, each pixel including a brightness level; 

Fig. 1b is a depiction of an exemplary sampling of a pattern image 
consisting of a set of edges, each edge including an angle of the brightness 
gradient at that edge; 

Fig. 2 is a flowchart of an embodiment of the method of the invention; 

Fig. 2a is a flowchart of the step of generating a set of partitions of Fig.2; 

Fig. 2b is a is a flowchart of the step of scoring the partitions of Fig. 2; 

Fig. 3a is a sample image to be subdivided; 

Fig. 3b is a set of feature points resulting from an analysis of the sample 
image of Fig. 3a; 

Fig. 3c is a first candidate partition of the set of feature points of Fig. 3b; 
Fig. 3d is a second candidate partition of the set of feature points of Fig. 

3b; 

Fig. 4a is an example of a set of feature points, each point having only 
position information; and 

Fig. 4b is a fully connected graph of the set of feature points of Fig. 4a, 
including links having weights attached. 
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DETAILED DESCRIPTION 

The method of the invention uses information present in the pattern itself 
as the basis for dividing the pattern into smaller parts called sub-patterns. 
Pattern information is discretized into 'feature points," which are points in the 
pattern where there is some feature, such as an edge, that can be used to 
effectively find the pattern. Many search methods are already based on such 
feature points, such as feature-based search methods, some of which may also 
be called "geometric" search methods. These feature points are then associated 
in small groups based on proximity to each other and other factors explained 
below, but without regard to such artificial means as grid-lines. The result is a 
plurality of sub-patterns where each sub-pattern contains a similar amount of 
information, where each sub-pattern has enough information to be located with a 
feature-based search (typically based on approximately 12 to 30 feature points, 
at differing angles), and where that information has also been pre-evaluated as 
being useful and particularly adapted for running feature-based searches. Less 
than 12 feature points may not be enough information for known feature-based 
searches, and more than 30 feature points can be usefully subdivided into two 
smaller sub-groups. 

An embodiment of the method of the invention takes as input either an 
image of a pattern 200 or an abstract description of a pattern 210 (e.g. a circle of 
radius 20 at a point 15,15), either of which can be analyzed so as to provide a 
representative feature list 220, (where a feature is a point of interest in the 
pattern, possibly augmented by other information such as a gradient angle or 
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magnitude), by any known feature extraction method, or more advantageously by 
using the two methods described herein below. 

One preferred embodiment of this step 220 is to sample the pattern image 
1 00 so as to provide a regular array of 2D points 110, each of which includes the 
image brightness level 120 at that point. Figure 1a shows nine (9) such point 
features, each represented by a circle 1 1 0 containing a brightness level 120. 
Another preferred embodiment of 220 is to convert the image 100 into a list of 
interesting edges 130 using any known edge extraction method to do so, and 
sample those edges into edge feature points 140 that include the angle 150 of 
the brightness gradient (or the angle of the edge) at that point 140. 

Alternatively, if some other method or tool has described the pattern as a 
list of features already, those features can be used directly 230, without the need 
for the pattern processing, such as described above. Note that features are 
typically 2D image points, but one skilled in the art can readily see that this 
method works identically with points of any dimensionality, such as 3D. 

Using the feature list (from either step 220 or 230), step 240 generates a 
set of partitions of the list, where each partition includes a portion of the list. 
Each partition might have only a single sub-list, or it might have many sub-lists. 
Certain features from the original list might not appear on any sub-list, or a 
feature or features might appear on multiple sub-lists. Any method known in the 



COGNEX CONFIDENTIAL 



12 



C03-002 

art can be used to generate this set of partitions, such as any clustering 
algorithm, or any spatial subdivision algorithm. As an example, Figure 3a shows 
a sample "image" 300 to be subdivided. Figure 3b shows a sample breakdown 
of the image into feature points 310, as described above. Figures 3c and 3d 
show two different candidate partitions of that list of features, where the black 
circles 320 are partitioned into one list, and the white circles 330 into another. 
The best methods are those that tend to yield sub-lists that are spatially small, 
that is, where features that are near each other tend to be in the same sub-list. 

A preferred embodiment for this step 240 is the "Normalized Cut" 
algorithm 246, as published in the article "Normalized Cuts and Image 
Segmentation" by Shi and Malik, 1997, herein incorporated by reference. To 
utilize this algorithm, the feature list is input 242 for conversion 244 into a 
weighted graph. The weighted graph is constructed by fully connecting the 
feature points to make a graph, and then setting the weights on each link based 
on the distance between those represented feature points. 

Note that some weights might be zero, in which case those links can be 
ignored, and the fully connected graph is equivalent to a sparsely connected 
graph with those zero-weight links missing. This does not affect the method, 
though it may allow a particular implementation to be more efficient if it can take 
advantage of the smaller number of links. 

Optionally, the weights can be set based on other values such as similarity 
of angle, magnitude, or any other values that may be available as part of the 
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features themselves. Larger weights represent a pair of points that should stay 
in the same sub-lists together, while smaller weights indicate points that can be 
broken apart more easily. Therefore, in a preferred embodiment, weights 
decrease as the distance between feature points increases. Exact formulas, and 
the weightings of other feature characteristics, will vary based on the pattern 
domain, but any formula can be used and still remain within the spirit of the 
invention. The Normalized Cut method is one that generates multiple partitions 
which are pairs of sub-lists, where each node (or feature) appears in exactly one 
of the two lists. These partitions are generated in such a way that the total 
weight of links between elements of the two sub-lists is small, and the total 
weight of links between elements within each sub-list is large. This nicely meets 
the criteria recognized by the invention for a good partition, when weights are 
high for features spatially near to each other. 

Figure 4a is an example of a set of five feature points 400 each point 
having only position information. Figure 4b shows a fully connected graph, 
where the links 410 between nodes 420 have weights 430 attached, with higher 
weights 440 on links between nodes near each other, and lower weights 450 on 
lines between more mutually distant nodes. When "Normalized Cut" is applied to 
this graph using the methods as published in the article cited above, it yields a 
list of partitions, each of which breaks the graph into two sub-graphs, one of 
which may be empty. These sub-graphs can be directly converted back into 
feature sub-lists, since each node in the graph represents exactly one feature 
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point. In the example of Figure 4b, the three nodes on the left will likely become 
one sub-list, and the two nodes on the right will become another sub-list, as the 
links between those sets of nodes have much lower weights than the links within 
each set. 

A scoring function 250 is then applied to each candidate partition. The 
scoring function can take into account a number of factors. One embodiment of 
this function scores partitions based on the spatial coherence of the features in 
each sub-list, where "spatial coherence" refers to features that have positions 
near to each other, relative to the distances of other features. Since the node 
weights used for the preferred embodiment of the partitioning method are higher 
for features that are close to one another, those weights can easily be summed 
and used in this embodiment. Another embodiment determines the score based 
on overall spatial size of the area spanned by the feature points (e.g. as 
represented by the smallest bounding box that includes all the feature points) 
included in each sub-list. Another embodiment determines the score based on 
the number of feature points in each sub-list. A preferred embodiment of the 
function uses the total amount of weight in links "cut" by the partition algorithm 
(for the "Normalized Cut" partitioning method described above), and another 
preferred embodiment builds a search pattern 254 from each feature sub-set 252 
and bases the score on the overall "suitability" (discussed below) of the sub-list 
as a search pattern on the original image 255. A preferred embodiment of this 
function is one that takes into account all of these elements 256 in determining 
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the score. In this final embodiment, each scoring method is applied to the sub- 
lists to determine a set of scores. Those scores are combined, either by simply 
summing them or by computing a weighted average based on the applicability of 
each individual method in the target domain or by any other reasonable method, 
and the total score is used. 

The last factor mentioned, that of "suitability" as a search pattern, depends 
on the particular algorithm that might later be used to do searching using the 
result of this invention. However, the "suitability" metric will likely include 
"degeneracy". For example, a sub-list where all points lie on the same line is 
probably not a suitable search pattern, nor is one where they lie on the same 
circle if the search will involve rotational uncertainty. The "suitability" metric will 
also likely include "redundancy" within the original pattern. For example, a set of 
features that can be matched at multiple locations, multiple angles, and/or 
multiple scales, etc., within the original pattern will probably yield undesirable and 
possibly unacceptable confusion during a search. 

A preferred embodiment of this metric is to run the search algorithm itself 
on the training image - if that search algorithm is known at the time this method 
is used. If the search returns only one match, the prospective sub-pattern scores 
well, i.e., is "suitable"; if it returns multiple matches or confusion about where the 
sub-pattern is located, the sub-pattern scores poorly, i.e., is "unsuitable". One 
skilled in the art will readily recognize that any search algorithm will have certain 
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factors that make some patterns easier to find (i.e., more "suitable*) than others, 
and this invention allows those factors, whatever they may be, to affect the 
"suitability" of the sub-pattern, and thereby affect the score of the partitions. 

There may be a score threshold 257 for this step, and that score threshold 
might be set outside the algorithm, or might be fixed within the algorithm, or 
might be based on a combination of multiple such values. If no partition has a 
score above this threshold, then the list of features is deemed to be one that 
cannot be usefully sub-divided. 

Otherwise, the top-scoring candidate partition is then used to create sub- 
lists from the original feature list 260. A partition specifies which feature points 
go into which sub-lists. The set of sub-lists generated by this step, if any, is 
returned as the output of the algorithm 270. One skilled in the art can readily see 
that these output sub-lists may then be returned to the algorithm as a pre- 
generated feature list 230, in which case the sub-division can be further refined 
by repeated, recursive application of this method. 

Other variants and embodiments will occur to those skilled in the art, 
without departing from the spirit and scope of the invention. Accordingly, the 
invention is not intended to be limited by the detailed description, except as set 
forth in the following claims. 
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